Conventionally known is a system in which an image processing apparatus is utilized as a visual sensor for an automatic machine, such as a robot, so that the deviations of the position and attitude of the object of operation detected by means of the image processing apparatus are detected. The data indicative of these deviations of the position and attitude are delivered as correction data to a control device of the automatic machine; and the control device of the automatic machine performs a specified operation while correcting the operating position in accordance with the correction data. In this case, the operation for the position correction cannot be performed normally unless a coordinate system for an image of the object of operation to be detected by the image processing apparatus and a coordinate system for the operation of the automatic machine are integrated. Accordingly, a process is needed to combine the respective coordinate systems of the image processing apparatus and the control device of the automatic machine.
In order to correct the operation of the automatic machine, such as a robot, in accordance with the correction data indicative of the deviations of the position and attitude of the object of operation detected by the image processing apparatus, it is necessary to transform pixel value data (data in a camera coordinate system) on the image, picked up by a camera of the image processing apparatus, into data in a sensor coordinate system, which is common to the image processing apparatus and the control device of the automatic machine, and also into data in a work coordinate system for the actual drive control of the automatic machine. These coordinate transformations for the data require previous definitions of the respective coordinate axes of the individual coordinate systems. In general, this series of operations is referred to as calibration.
The calibration work comprises operations including:
(1) setting on the visual sensor side and data setting essential to the coordinate definitions;
(2) coordinate definition for the visual sensor;
(3) coordinate definition for the control device of the robot (automatic machine); and
(4) combining the coordinates for the visual sensor and the robot control device.
In executing two-dimensional calibration associated with the image processing apparatus and the robot, for example, the image processing apparatus combines the camera coordinates and sensor coordinates (operations (1) and (2) of the calibration work) by setting the positional relationships between the camera coordinate axis and the sensor coordinates, axis names of the sensor coordinates, pixel size, etc., and then defines the work coordinates for the robot operation (coordinate definition on the robot side described in (3)).
In the aforesaid operation (4), a body which can be recognized by the image processing apparatus, e.g., a jig with a circular object thereon, is first prepared for use and mounted on the robot. Then, the jig on the robot is moved to the origin of the sensor coordinates recognized by the image processing apparatus, and its position (position with respect to the work coordinates for the robot) is stored in the control device of the robot. Then, the robot is moved to specified points on the X and Y axes of the sensor coordinates, and the respective positions of these points are stored individually. Subsequently, these stored data are displayed on a display unit on the robot side, and their respective values are entered in the image processing apparatus. Thereupon, the relationships between the work coordinates for the robot and the sensor coordinates are obtained, and the sensor coordinates recognized by the robot are squared with the sensor coordinates recognized by the image processing apparatus to complete the calibration work.
This calibration work is a highly expert task, which requires very high accuracy and complicated operation processes.
Known examples of this calibration work, besides the aforementioned two-dimensional calibration, include three-dimensional calibration such that a three-dimensional position is detected by picking up the object of operation by three cameras. Also known is a method in which the two- and three-dimensional calibrations are effected by using dot patterns.
Normally, the calibration work described above need not be repeated if it has been executed once during the construction of the work system including the image processing apparatus, robot, etc.
If errors increase for any reasons, such as deviation of the camera, change of the optical environment, etc., the calibration should be executed again. In this case, the operations (1) to (4) of the calibration work must be repeated, thus requiring time, labor, and service of an expert.